Systems and methods related to registration for image guided surgery

ABSTRACT

A system is configured to perform operations includes accessing a set of model points of a model of an anatomic structure of a patient, the model points being associated with a model space. A set of measured points of the anatomic structure of the patient are collected, the measured points being associated with a patient space. The set of model points are registered to the set of measured points using a first set of initial parameters to generate a first transformation. One or more sets of perturbed initial parameters are generated based on the first set of initial parameters. One or more perturbed registration processes are performed to register the set of model points to the set of measured points using the one or more sets of perturbed initial parameters respectively to generate corresponding perturbed transformations. A registration quality indicator is generated based on the first transformation and the one or more perturbed transformations.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application62/670,530 filed May 11, 2018, which is incorporated by reference hereinin its entirety.

FIELD

The present disclosure is directed to systems and methods for conductingan image-guided procedure, and more particularly to systems and methodsfor using registered real-time images and prior-time anatomic imagesduring an image-guided procedure.

BACKGROUND

Minimally invasive medical techniques are intended to reduce the amountof tissue that is damaged during medical procedures, thereby reducingpatient recovery time, discomfort, and harmful side effects. Suchminimally invasive techniques may be performed through natural orificesin a patient anatomy or through one or more surgical incisions. Throughthese natural orifices or incisions an operator may insert minimallyinvasive medical instruments (including surgical, diagnostic,therapeutic, or biopsy instruments) to reach a target tissue location.To assist with reaching the target tissue location, the location andmovement of the medical instruments may be correlated with pre-operativeor intra-operative images of the patient anatomy. With the image-guidedinstruments correlated to the images, the instruments may navigatenatural or surgically created passageways in anatomic systems such asthe lungs, the colon, the intestines, the kidneys, the heart, thecirculatory system, or the like. However, usually an operator does nothave sufficient knowledge about the quality (e.g., accuracy,completeness, validity, consistency) of such correlation, which maycause uncertainty in the image-guided procedure.

Accordingly, it would be advantageous to provide improved registrationfor performing image-guided procedures.

SUMMARY

The embodiments of the invention are best summarized by the claims thatfollow the description.

In one illustrative embodiment, a method performed by a computing systemincludes accessing a set of model points of a model of an anatomicstructure of a patient, the model points being associated with a modelspace, collecting a set of measured points of the anatomic structure ofthe patient, the measured points being associated with a patient space,and registering the set of model points to the set of measured pointsusing a first set of initial parameters to generate a firsttransformation. The method further includes generating one or more setsof perturbed initial parameters based on the first set of initialparameters, performing one or more perturbed registration processes toregister the set of model points to the set of measured points using theone or more sets of perturbed initial parameters respectively togenerate one or more perturbed transformations; and generating aregistration quality indicator based on the first transformation and theone or more perturbed transformations.

In another illustrative embodiment, a method performed by a computingsystem includes accessing a set of model points of a model of ananatomic structure of a patient, the model points being associated witha model space, collecting a set of measured points of the anatomicstructure of the patient, the measured points being associated with apatient space, determining a region of interest in the model space, anddetermining a comparison region in the model space. The method furtherincludes registering a first set of model points in the region ofinterest to the set of measured points to generate a first number ofregion of interest transformations, registering a second set of themodel points in the comparison region to the set of measured points togenerate a first number of comparison region transformationscorresponding to the first number of region of interest transformationsrespectively, and generating a registration quality indicator for theregion of interest based on the first number of region of interesttransformations and the first number of comparison regiontransformations.

In another illustrative embodiment, a method performed by a computingsystem includes accessing a set of model points of a model of ananatomic structure of a patient, the model points being associated witha model space, collecting a set of measured points of the anatomicstructure of the patient, the measured points being associated with apatient space, and registering the set of model points to the set ofmeasured points to generate a first transformation. The method furtherincludes performing a local registration quality analysis for a regionof interest to generate a local registration quality indicator,generating a modified model space based on the registration qualityindicator, and registering a subset set of model points associated withthe modified model space to the set of measured points to generate asecond transformation.

In another illustrative embodiment, a non-transitory machine-readablemedium comprising a plurality of machine-readable instructions which,when executed by one or more processors, are adapted to cause the one ormore processors to perform a method. The method includes accessing a setof model points of a model of an anatomic structure of a patient, themodel points being associated with a model space, collecting a set ofmeasured points of the anatomic structure of the patient, the measuredpoints being associated with a patient space, and registering the set ofmodel points to the set of measured points using a first set of initialparameters to generate a first transformation. The method furtherincludes generating one or more sets of perturbed initial parametersbased on the first set of initial parameters, performing one or moreperturbed registration processes to register the set of model points tothe set of measured points using the one or more sets of perturbedinitial parameters respectively to generate one or more perturbedtransformations, and generating a registration quality indicator basedon the first transformation and the one or more perturbedtransformations.

In another illustrative embodiment, a non-transitory machine-readablemedium comprising a plurality of machine-readable instructions which,when executed by one or more processors, are adapted to cause the one ormore processors to perform a method. The method includes accessing a setof model points of a model of an anatomic structure of a patient, themodel points being associated with a model space, collecting a set ofmeasured points of the anatomic structure of the patient, the measuredpoints being associated with a patient space, determining a region ofinterest in the model space, and determining a comparison region in themodel space. The method further includes registering a first set ofmodel points in the region of interest to the set of measured points togenerate a first number of region of interest transformations,registering a second set of the model points in the comparison region tothe set of measured points to generate a first number of comparisonregion transformations corresponding to the first number of region ofinterest transformations respectively, and generating a registrationquality indicator for the region of interest based on the first numberof region of interest transformations and the first number of comparisonregion transformations.

In another illustrative embodiment, a non-transitory machine-readablemedium comprising a plurality of machine-readable instructions which,when executed by one or more processors, are adapted to cause the one ormore processors to perform a method. The method includes accessing a setof model points of a model of an anatomic structure of a patient, themodel points being associated with a model space, collecting a set ofmeasured points of the anatomic structure of the patient, the measuredpoints being associated with a patient space, and registering the set ofmodel points to the set of measured points to generate a firsttransformation. The method further includes performing a localregistration quality analysis for a region of interest to generate alocal registration quality indicator, generating a modified model spacebased on the registration quality indicator, and registering a subsetset of model points associated with the modified model space to the setof measured points to generate a second transformation.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory innature and are intended to provide an understanding of the presentdisclosure without limiting the scope of the present disclosure. In thatregard, additional aspects, features, and advantages of the presentdisclosure will be apparent to one skilled in the art from the followingdetailed description.

BRIEF DESCRIPTIONS OF THE DRAWINGS

FIG. 1 is a simplified diagram of a teleoperated medical systemaccording to some embodiments.

FIG. 2A is a simplified diagram of a medical instrument system accordingto some embodiments,

FIG. 2B is a simplified diagram of a medical instrument with an extendedmedical tool according to some embodiments.

FIGS. 3A and 3B are simplified diagrams of side views of a patientcoordinate space including a medical instrument mounted on an insertionassembly according to some embodiments.

FIGS. 4A, 4B, 4C, and 4D illustrate the distal end of the medicalinstrument system of FIGS. 2, 3A, 3B, during insertion within a humanlung according to some embodiments.

FIG. 5 is a flowchart illustrating a method of an image-guided surgicalprocedure or a portion thereof according to some embodiments.

FIGS. 6A, 6B, and 6C illustrate steps in segmentation processes thatgenerate a model of a human lung for registration according to someembodiments.

FIG. 7 is a flow chart providing a method for performing a globalregistration quality analysis according to some embodiments.

FIG. 8 is a flow chart providing a method for performing a localregistration quality analysis according to some embodiments.

FIG. 9 is a flow chart providing a method for performing re-registrationbased on registration quality analysis according to some embodiments.

FIG. 10 illustrates a display stage of a re-registration techniqueaccording to some embodiments.

FIG. 11 illustrates a display stage of a re-registration techniqueaccording to some embodiments.

Embodiments of the present disclosure and their advantages are bestunderstood by, referring to the detailed description that follows. Itshould be appreciated that like reference numerals are used to identifylike elements illustrated in one or more of the figures, whereinshowings therein are for purposes of illustrating embodiments of thepresent disclosure and not for purposes of limiting the same.

DETAILED DESCRIPTION

In the following description, specific details are set forth describingsome embodiments consistent with the present disclosure. Numerousspecific details are set forth in order to provide a thoroughunderstanding of the embodiments. It will be apparent, however, to oneskilled in the art that some embodiments may be practiced without someor all of these specific details. The specific embodiments disclosedherein are meant to be illustrative but not limiting. One skilled in theart may realize other elements that, although not specifically describedhere, are within the scope and the spirit of this disclosure. Inaddition, to avoid unnecessary repetition, one or more features shownand described in association with one embodiment may be incorporatedinto other embodiments unless specifically described otherwise or if theone or more features would make an embodiment non-functional.

In some instances well known methods, procedures, components, andcircuits have not been described in detail so as not to unnecessarilyobscure aspects of the embodiments.

This disclosure describes various instruments and portions ofinstruments in terms of their state in three-dimensional space. As usedherein, the term “position” refers to the location of an object or aportion of an object in a three-dimensional space (e.g., three degreesof translational freedom along Cartesian x-, y-, and z-coordinates). Asused herein, the term “orientation” refers to the rotational placementof an object or a portion of an object (three degrees of rotationalfreedom—e.g., roll, pitch, and yaw). As used herein, the term “pose”refers to the position of an object or a portion of an object in atleast one degree of translational freedom and to the orientation of thatobject or portion of the object in at least one degree of rotationalfreedom (up to six total degrees of freedom). As used herein, the term“shape” refers to a set of poses, positions, or orientations measuredalong an object.

FIG. 1 is a simplified diagram of a teleoperated medical system 100according to some embodiments. In some embodiments, teleoperated medicalsystem 100 may be suitable for use in, for example, surgical,diagnostic, therapeutic, or biopsy procedures. While some embodimentsare provided herein with respect to such procedures, any reference tomedical or surgical instruments and medical or surgical methods isnon-limiting. The systems, instruments, and methods described herein maybe used for animals, human cadavers, animal cadavers, portions of humanor animal anatomy, non-surgical diagnosis, as well as for industrialsystems and general robotic or teleoperational systems.

As shown in FIG. 1 , medical system 100 generally includes a manipulatorassembly 102 for operating a medical instrument 104 in performingvarious procedures on a patient P. The manipulator assembly 102 may beteleoperated, non-teleoperated, or a hybrid teleoperated andnon-teleoperated assembly with select degrees of freedom of motion thatmay be motorized and/or teleoperated and select degrees of freedom ofmotion that may be non-motorized and/or non-teleoperated. Manipulatorassembly 102 is mounted to or near an operating table T. A masterassembly 106 allows an operator (e.g., a surgeon, a clinician, or aphysician as illustrated in FIG. 1 ) 0 to view the interventional siteand to control manipulator assembly 102.

Master assembly 106 may be located at an operator console which isusually located in the same room as operating table T, such as at theside of a surgical table on which patient P is located. However, itshould be understood that operator O can be located in a different roomor a completely different building from patient P. Master assembly 106generally includes one or more control devices for controllingmanipulator assembly 102. The control devices may include any number ofa variety of input devices, such as joysticks, trackballs, data gloves,trigger-guns, hand-operated controllers, voice recognition devices, bodymotion or presence sensors, and/or the like. To provide operator 1) astrong sense of directly controlling instruments 104 the control devicesmay be provided with the same degrees of freedom as the associatedmedical instrument 104. In this manner, the control devices provideoperator O with telepresence or the perception that the control devicesare integral with medical instruments 104.

In some embodiments, the control devices may have more or fewer degreesof freedom than the associated medical instrument 104 and still provideoperator O with telepresence. In some embodiments, the control devicesmay optionally be manual input devices which move with six degrees offreedom, and which may also include an actuatable handle for actuatinginstruments (for example, for closing grasping jaws, applying anelectrical potential to an electrode, delivering a medicinal treatment,and/or the like).

Manipulator assembly 102 supports medical instrument 104 and may includea kinematic structure of one or more non-servo controlled links (e.g.,one or more links that may be manually positioned and locked in place,generally referred to as a set-up structure), and/or one or more servocontrolled links (e.g. one more links that may be controlled in responseto commands from the control system), and a manipulator. Manipulatorassembly 102 may optionally include a plurality of actuators or motorsthat drive inputs on medical instrument 104 in response to commands fromthe control system (e.g., a control system 112). The actuators mayoptionally include drive systems that when coupled to medical instrument104 may advance medical instrument 104 into a naturally or surgicallycreated anatomic orifice. Other drive systems may move the distal end ofmedical instrument 104 in multiple degrees of freedom, which may includethree degrees of linear motion (e.g., linear motion along the X, Y, ZCartesian axes) and in three degrees of rotational motion (e.g.,rotation about the X, Y, Z Cartesian axes), Additionally, the actuatorscan be used to actuate an articulable end effector of medical instrument104 for grasping tissue in the jaws of a biopsy device and/or the like.Actuator position sensors such as resolvers, encoders, potentiometers,and other mechanisms may provide sensor data to medical system 100describing the rotation and orientation of the motor shafts. Thisposition sensor data may be used to determine motion of the objectsmanipulated by the actuators.

Teleoperated medical system 100 may include a sensor system 108 with oneor more sub-systems for receiving information about the instruments ofmanipulator assembly 102. Such sub-systems may include aposition/location sensor system (e.g., an electromagnetic (EM) sensorsystem); a shape sensor system for determining the position,orientation, speed, velocity, pose, and/or shape of a distal end and/orof one or more segments along a flexible body that may make up medicalinstrument 104; and/or a visualization system for capturing images fromthe distal end of medical instrument 104.

Teleoperated medical system 100 also includes a display system 110 fordisplaying an image or representation of the surgical site and medicalinstrument 104 generated by sub-systems of sensor system 108. Displaysystem 110 and master assembly 106 may be oriented so operator O cancontrol medical instrument 104 and master assembly 106 with theperception of telepresence.

In some embodiments, medical instrument 104 may have a visualizationsystem (discussed in more detail below), which may include a viewingscope assembly that records a concurrent or real-time image of asurgical site and provides the image to the operator or operator Othrough one or more displays of medical system 100, such as one or moredisplays of display system 110. The concurrent image may be, forexample, a two or three dimensional image captured by an endoscopepositioned within the surgical site. In some embodiments, thevisualization system includes endoscopic components that may beintegrally or removably coupled to medical instrument 104. However insome embodiments, a separate endoscope, attached to a separatemanipulator assembly may be used with medical instrument 104 to imagethe surgical site. The visualization system may be implemented ashardware, firmware, software or a combination thereof which interactwith or are otherwise executed by one or more computer processors, whichmay include the processors of a control system 112.

Display system 110 may also display an image of the surgical site andmedical instruments captured by the visualization system. In someexamples, teleoperated medical system 100 may configure medicalinstrument 104 and controls of master assembly 106 such that therelative positions of the medical instruments are similar to therelative positions of the eyes and hands of operator O. In this manneroperator O can manipulate medical instrument 104 and the hand control asif viewing the workspace in substantially true presence. By truepresence, it is meant that the presentation of an image is a trueperspective image simulating the viewpoint of a physician that isphysically manipulating medical instrument 104.

In some examples, display system 110 may present images of a surgicalsite recorded pre-operatively or intra-operatively using image data fromimaging technology such as, computed tomography (CT), magnetic resonanceimaging (MRI), fluoroscopy, thermography, ultrasound, optical coherencetomography (OCT), thermal imaging, impedance imaging, laser imaging,nanotube X-ray imaging, and/or the like. The pre-operative orintra-operative image data may be presented as two-dimensional,three-dimensional, or four-dimensional (including e.g., time based orvelocity based information) images and/or as images from models createdfrom the pre-operative or intra-operative image data sets.

In some embodiments, often for purposes of imaged guided surgicalprocedures, display system 110 may display a virtual navigational imagein which the actual location of medical instrument 104 is registered(i.e., dynamically referenced) with the preoperative or concurrentimages/model. This may be done to present the operator O with a virtualimage of the internal surgical site from a viewpoint of medicalinstrument 104. In some examples, the viewpoint may be from a tip ofmedical instrument 104. An image of the tip of medical instrument 104and/or other graphical or alphanumeric indicators may be superimposed onthe virtual image to assist operator (1) controlling medical instrument104. In some examples, medical instrument 104 may not be visible in thevirtual image.

In some embodiments, display system 110 may display a virtualnavigational image in which the actual location of medical instrument104 is registered with preoperative or concurrent images to present theoperator O with a virtual image of medical instrument 104 within thesurgical site from an external viewpoint. An image of a portion ofmedical instrument 104 or other graphical or alphanumeric indicators maybe superimposed on the virtual image to assist operator O in the controlof medical instrument 104. As described herein, visual representationsof data points may be rendered to display system 110. For example,measured data points, moved data points, registered data points, andother data points described herein may be displayed on display system110 in a visual representation. The data points may be visuallyrepresented in a user interface by a plurality of points or dots ondisplay system 110 or as a rendered model, such as a mesh or wire modelcreated based on the set of data points. In some examples, the datapoints may be color coded according to the data they represent. In someembodiments, a visual representation may be refreshed in display system110 after each processing operation has been implemented to alter datapoints.

Teleoperated medical system 100 may also include control system 112.Control system 112 includes at least one memory and at least onecomputer processor (not shown) for effecting control between medicalinstrument 104, master assembly 106, sensor system 108, and display,system 110. Control system 112 also includes programmed instructions(e.g., a non-transitory machine-readable medium storing theinstructions) to implement some or all of the methods described inaccordance with aspects disclosed herein, including instructions forproviding information to display system 110. While control system 112 isshown as a single block in the simplified schematic of FIG. 1 , thesystem may include two or more data processing circuits with one portionof the processing optionally being performed on or adjacent tomanipulator assembly 102, another portion of the processing beingperformed at master assembly 106, and/or the like. The processors ofcontrol system 112 may execute instructions comprising instructioncorresponding to processes disclosed herein and described in more detailbelow. Any of a wide variety of centralized or distributed dataprocessing architectures may be employed. Similarly, the programmedinstructions may be implemented as a number of separate programs orsubroutines, or they may be integrated into a number of other aspects ofthe teleoperational systems described herein. In one embodiment, controlsystem 112 supports wireless communication protocols such as Bluetooth,IrDA, HomeRF, IEEE 802.11, DECT, and Wireless Telemetry.

In some embodiments, control system 112 may receive force and/or torquefeedback from medical instrument 104. Responsive to the feedback,control system 112 may transmit signals to master assembly 106. In someexamples, control system 112 may transmit signals instructing one ormore actuators of manipulator assembly 102 to move medical instrument104. Medical instrument 104 may extend into an internal surgical sitewithin the body of patient P via openings in the body of patient P. Anysuitable conventional and/or specialized actuators may be used. In someexamples, the one or more actuators may be separate from, or integratedwith, manipulator assembly 102. In some embodiments, the one or moreactuators and manipulator assembly 102 are provided as part of ateleoperational cart positioned adjacent to patient P and operatingtable T.

Control system 112 may optionally further include a virtualvisualization system to provide navigation assistance to operator O whencontrolling medical instrument 104 during an image-guided surgicalprocedure. Virtual navigation using the virtual visualization system maybe based upon reference to an acquired preoperative or intraoperativedataset of anatomic passageways. The virtual visualization systemprocesses images of the surgical site imaged using imaging technologysuch as computerized tomography (CT), magnetic resonance imaging (MRI),fluoroscopy, thermography, ultrasound, optical coherence tomography(OCT), thermal imaging, impedance imaging, laser imaging, nanotube X-rayimaging, and/or the like. Software, which may be used in combinationwith manual inputs, is used to convert the recorded images intosegmented two dimensional or three dimensional composite representationof a partial or an entire anatomic organ or anatomic region. An imagedata set is associated with the composite representation. The compositerepresentation and the image data set describe the various locations andshapes of the passageways and their connectivity. The images used togenerate the composite representation may be recorded preoperatively orintra-operatively during a clinical procedure. In some embodiments, avirtual visualization system may use standard representations (i.e., notpatient specific) or hybrids of a standard representation and patientspecific data. The composite representation and any virtual imagesgenerated by the composite representation may represent the staticposture of a deformable anatomic region during one or more phases ofmotion (e.g., during an inspiration/expiration cycle of a lung).

During a virtual navigation procedure, sensor system 108 may be used tocompute an approximate location of medical instrument 104 with respectto the anatomy of patient P. The location can be used to produce bothmacro-level (external) tracking images of the anatomy of patient P andvirtual internal images of the anatomy of patient P. The system mayimplement one or more electromagnetic (EM) sensor, fiber optic sensors,and/or other sensors to register and display a medical implementtogether with preoperatively recorded surgical images, such as thosefrom a virtual visualization system. For example, PCT Publication WO2016/191298 (published Dec. 1, 2016) (disclosing “Systems and Methods ofRegistration for image Guided Surgery”), which is incorporated byreference herein in its entirety, discloses such one system.Teleoperated medical system 100 may further include optional operationsand support systems (not shown) such as illumination systems, steeringcontrol systems, irrigation systems, and/or suction systems. In someembodiments, teleoperated medical system 100 may include more than onemanipulator assembly and/or more than one master assembly. The exactnumber of teleoperational manipulator assemblies will depend on thesurgical procedure and the space constraints within the operating room,among other factors. Master assembly 106 may be collocated or they maybe positioned in separate locations. Multiple master assemblies allowmore than one operator to control one or more teleoperationalmanipulator assemblies in various combinations.

FIG. 2A is a simplified diagram of a medical instrument system 200according to some embodiments. In some embodiments, medical instrumentsystem 200 may be used as medical instrument 104 in an image-guidedmedical procedure performed with teleoperated medical system 100. Insome examples, medical instrument system 200 may be used fornon-teleoperational exploratory procedures or in procedures involvingtraditional manually operated medical instruments, such as endoscopy.Optionally medical instrument system 200 may be used to gather (i.e.,measure) a set of data points corresponding to locations within anatomicpassageways of a patient, such as patient P.

Medical instrument system 200 includes elongate device 202, such as aflexible catheter, coupled to a drive unit 204. Elongate device 202includes a flexible body 216 having proximal end 217 and distal end ortip portion 218. In some embodiments, flexible body 216 has anapproximately 3 mm outer diameter. Other flexible body outer diametersmay be larger or smaller.

Medical instrument system 200 further includes a tracking system 230 fordetermining the position, orientation, speed, velocity, pose, and/orshape of distal end 218 and/or of one or more segments 224 alongflexible body 216 using one or more sensors and/or imaging devices asdescribed in further detail below. The entire length of flexible body216, between distal end 218 and proximal end 217, may be effectivelydivided into segments 224. Tracking system 230 may optionally beimplemented as hardware, firmware, software or a combination thereofwhich interact with or are otherwise executed by one or more computerprocessors, which may include the processors of control system 112 inFIG. 1 .

Tracking system 230 may optionally track distal end 218 and/or one ormore of the segments 224 using a shape sensor 222. Shape sensor 222 mayoptionally include an optical fiber aligned with flexible body 216(e.g., provided within an interior channel (not shown) or mountedexternally). In one embodiment, the optical fiber has a diameter ofapproximately 200 μm. In other embodiments, the dimensions may be largeror smaller. The optical fiber of shape sensor 222 forms a fiber opticbend sensor for determining the shape of flexible body 216. In onealternative, optical fibers including Fiber Bragg Gratings (FBGs) areused to provide strain measurements in structures in one or moredimensions. Various systems and methods for monitoring the shape andrelative position of an optical fiber in three dimensions are describedin U.S. patent application Ser. No. 11/180,389 (filed Jul. 13, 2005)(disclosing “Fiber optic position and shape sensing device and methodrelating thereto”); U.S. patent application Ser. No. 12/047,056 (filedon Jul. 16, 2004) (disclosing “Fiber-optic shape and relative positionsensing”); and U.S. Pat. No. 6,389,187 (filed on Jun. 17, 1998)(disclosing “Optical Fibre Bend Sensor”), which are all incorporated byreference herein in their entireties. Sensors in some embodiments mayemploy other suitable strain sensing techniques, such as Rayleighscattering, Raman scattering, Brillouin scattering, and Fluorescencescattering. In some embodiments, the shape of the elongate device may bedetermined using other techniques. For example, a history of the distalend pose of flexible body 216 can be used to reconstruct the shape offlexible body 216 over the interval of time. In some embodiments,tracking system 230 may optionally and/or additionally track distal end218 using a position sensor system 220, Position sensor system 220 maybe a component of an EM sensor system with position sensor system 220including one or more conductive coils that may be subjected to anexternally generated electromagnetic field. Each coil of the EM sensorsystem then produces an induced electrical signal having characteristicsthat depend on the position and orientation of the coil relative to theexternally generated electromagnetic field. In some embodiments,position sensor system 220 may be configured and positioned to measuresix degrees of freedom, e.g., three position coordinates X, Y, Z andthree orientation angles indicating pitch, yaw, and roll of a base pointor five degrees of freedom, e.g., three position coordinates X, Y, Z andtwo orientation angles indicating pitch and yaw of a base point. Furtherdescription of a position sensor system is provided in U.S. Pat. No.6,380,732 (filed Aug. 11, 1999) (disclosing “Six-Degree of FreedomTracking System having a Passive Transponder on the Object BeingTracked”), which is incorporated by reference herein in its entirety.

In some embodiments, tracking system 230 may alternately and/oradditionally rely on historical pose, position, or orientation datastored for a known point of an instrument system along a cycle ofalternating motion, such as breathing. This stored data may be used todevelop shape information about flexible body 216. In some examples, aseries of positional sensors (not shown), such as electromagnetic (EM)sensors similar to the sensors in position sensor 220 may be positionedalong flexible body 216 and then used for shape sensing. In someexamples, a history of data from one or more of these sensors takenduring a procedure may be used to represent the shape of elongate device202, particularly if an anatomic passageway is generally static.

Flexible body 216 includes a channel 221 sized and shaped to receive amedical instrument 226. FIG. 2B is a simplified diagram of flexible body216 with medical instrument 226 extended according to some embodiments.In some embodiments, medical instrument 226 may be used for proceduressuch as surgery, biopsy, ablation, illumination, irrigation, or suction.Medical instrument 226 can be deployed through channel 221 of flexiblebody 216 and used at a target location within the anatomy. Medicalinstrument 226 may include, for example, image capture probes, biopsyinstruments, laser ablation fibers, and/or other surgical, diagnostic,or therapeutic tools. Medical tools may include end effectors having asingle working member such as a scalpel, a blunt blade, an opticalfiber, an electrode, and/or the like. Other end effectors may include,for example, forceps, graspers, scissors, clip appliers, and/or thelike. Other end effectors may further include electrically activated endeffectors such as electrosurgical electrodes, transducers, sensors,and/or the like. In various embodiments, medical instrument 226 is abiopsy instrument, which may be used to remove sample tissue or asampling of cells from a target anatomic location. Medical instrument226 may be used with an image capture probe also within flexible body216. In various embodiments, medical instrument 226 may be an imagecapture probe that includes a distal portion with a stereoscopic ormonoscopic camera at or near distal end 218 of flexible body 216 forcapturing images (including video images) that are processed by avisualization system 231 for display and/or provided to tracking system230 to support tracking of distal end 218 and/or one or more of thesegments 224. The image capture probe may include a cable coupled to thecamera for transmitting the captured image data. In some examples, theimage capture instrument may be a fiber-optic bundle, such as afiberscope, that couples to visualization system 231. The image captureinstrument may be single or multi-spectral, for example capturing imagedata in one or more of the visible, infrared, and/or ultravioletspectrums. Alternatively, medical instrument 226 may itself be the imagecapture probe. Medical instrument 226 may be advanced from the openingof channel 221 to perform the procedure and then retracted back into thechannel when the procedure is complete. Medical instrument 226 may beremoved from proximal end 217 of flexible body 216 or from anotheroptional instrument port (not shown) along flexible body 216.

Medical instrument 226 may additionally house cables, linkages, or otheractuation controls (not shown) that extend between its proximal anddistal ends to controllably the bend distal end of medical instrument226. Steerable instruments are described in detail in U.S. Pat. No.7,316,681 (filed on Oct. 4, 2005) (disclosing “Articulated SurgicalInstrument for Performing Minimally invasive Surgery with EnhancedDexterity and Sensitivity”) and U.S. patent application Ser. No.12/286,644 (filed Sep. 30, 2008) (disclosing “Passive Preload andCapstan Drive for Surgical Instruments”), which are incorporated byreference herein in their entireties.

Flexible body 216 may also house cables, linkages, or other steeringcontrols (not shown) that extend between drive unit 204 and distal end218 to controllably bend distal end 218 as shown, for example, by brokendashed line depictions 219 of distal end 218. In some examples, at leastfour cables are used to provide independent “up-down” steering tocontrol a pitch of distal end 218 and “left-right” steering to control ayaw of distal end 281. Steerable elongate devices are described indetail in U.S. patent application Ser. No. 13/274,208 (filed Oct. 14,2011) (disclosing “Catheter with Removable Vision Probe”), which isincorporated by reference herein in its entirety. In embodiments inwhich medical instrument system 200 is actuated by a teleoperationalassembly, drive unit 204 may include drive inputs that removably coupleto and receive power from drive elements, such as actuators, of theteleoperational assembly. In some embodiments, medical instrument system200 may include gripping features, manual actuators, or other componentsfor manually controlling the motion of medical instrument system 200.Elongate device 202 may be steerable or, alternatively, the system maybe non-steerable with no integrated mechanism for operator control ofthe bending of distal end 218. In some examples, one or more lumens,through which medical instruments can be deployed and used at a targetsurgical location, are defined in the walls of flexible body 216.

In some embodiments, medical instrument system 200 may include aflexible bronchial instrument, such as a bronchoscope or bronchialcatheter, for use in examination, diagnosis, biopsy, or treatment of alung. Medical instrument system 200 is also suited for navigation andtreatment of other tissues, via natural or surgically created connectedpassageways, in any of a variety of anatomic systems, including thecolon, the intestines, the kidneys and kidney calices, the brain, theheart, the circulatory system including vasculature, and/or the like.

The information from tracking system 230 may be sent to a navigationsystem 232 where it is combined with information from visualizationsystem 231 and/or the preoperatively obtained models to provide thephysician or other operator with real-time position information. In someexamples, the real-time position information may be displayed on displaysystem 110 of FIG. 1 for use in the control of medical instrument system200. In some examples, control system 116 of FIG. 1 may utilize theposition information as feedback for positioning medical instrumentsystem 200. Various systems for using fiber optic sensors to registerand display a surgical instrument with surgical images are provided inU.S. patent application Ser. No. 13/107,562, filed May 13, 2011,disclosing, “Medical System Providing Dynamic Registration of a Model ofan Anatomic Structure for Image-Guided Surgery,” PCT Publication WO2016/1033596 (filed May 20, 2016) (disclosing “Systems and Methods ofRegistration for Image Guided Surgery”), and PCT Publication WO2016/164311 (filed. Apr. 4, 2016) (disclosing “Systems and Methods ofRegistration Compensation in Image Guided Surgery”), which areincorporated by reference herein in their entirety.

In some examples, medical instrument system 200 may be teleoperatedwithin medical system 100 of FIG. 1 . In some embodiments, manipulatorassembly 102 of FIG. 1 may be replaced by direct operator control. Insome examples, the direct operator control may include various handlesand operator interfaces for hand-held operation of the instrument.

FIGS. 3A and 3B are simplified diagrams of side views of a patientcoordinate space including a medical instrument mounted on an insertionassembly according to some embodiments. As shown in FIGS. 3A and 3B, asurgical environment 300 includes a patient P is positioned on the tableT of FIG. 1 . Patient may be stationary within the surgical environmentin the sense that gross patient movement is limited by sedation,restraint, and/or other means. Cyclic anatomic motion includingrespiration and cardiac motion of patient P may continue, unless patientis asked to hold his or her breath to temporarily suspend respiratorymotion. Accordingly, in some embodiments, data may be gathered at aspecific, phase in respiration, and tagged and identified with thatphase. In some embodiments, the phase during which data is collected maybe inferred from physiological information collected from patient P.Within surgical environment 300, a point gathering instrument 304 iscoupled to an instrument carriage 306. In some embodiments, pointgathering instrument 304 may use EM sensors, shape-sensors, and/or othersensor modalities. Instrument carriage 306 is mounted to an insertionstage 308 fixed within surgical environment 300. Alternatively,insertion stage 308 may be movable but have a known location (e.g., viaa tracking sensor or other tracking device) within surgical environment300. Instrument carriage 306 may be a component of a manipulatorassembly (e.g., manipulator assembly 102) that couples to pointgathering instrument 304 to control insertion motion (i.e., motion alongthe A axis) and, optionally, motion of a distal end 318 of an elongatedevice 310 in multiple directions including yaw, pitch, and roll.Instrument carriage 306 or insertion stage 308 may include actuators,such as servomotors, (not shown) that control motion of instrumentcarriage 306 along insertion stage 308.

Elongate device 310 is coupled to an instrument body 312. Instrumentbody 312 is coupled and fixed relative to instrument carriage 306. Insome embodiments, an optical fiber shape sensor 314 is fixed at aproximal point 316 on instrument body 312. In some embodiments, proximalpoint 316 of optical fiber shape sensor 314 may be movable along withinstrument body 312 but the location of proximal point 316 may be known(e.g., via a tracking sensor or other (racking device). Shape sensor 314measures a shape from proximal point 316 to another point such as distalend 318 of elongate device 310. Point gathering instrument 304 may besubstantially similar to medical instrument system 200.

A position measuring device 320 provides information about the positionof instrument body 312 as it moves on insertion stage 308 along aninsertion axis A. Position measuring device 320 may include resolvers,encoders, potentiometers, and/or other sensors that determine therotation and/or orientation of the actuators controlling the motion ofinstrument carriage 306 and consequently the motion of instrument body312. In some embodiments, insertion stage 308 is ear. In someembodiments, insertion stage 308 may be curved or have a combination ofcurved and linear sections.

FIG. 3A shows instrument body 312 and instrument carriage 306 in aretracted position along insertion stage 308. In this retractedposition, proximal point 316 is at a position Lo on axis A. In thisposition along insertion stage 308 an A component of the location ofproximal point 316 may be set to a zero and/or another reference valueto provide a base reference to describe the position of instrumentcarriage 306, and thus proximal point 316, on insertion stage 308. Withthis retracted position of instrument body 312 and instrument carriage306, distal end 318 of elongate device 310 may be positioned just insidean entry orifice of patient P. Also in this position, position measuringdevice 320 may be set to a zero and/or the another reference value(e.g., I=0). In FIG. 3B, instrument body 312 and instrument carriage 306have advanced along the linear track of insertion stage 308 and distalend 318 of elongate device 310 has advanced into patient P. In thisadvanced position, the proximal point 316 is at a position Li on theaxis A. In some examples, encoder and/or other position data from one ormore actuators controlling movement of instrument carriage 306 alonginsertion stage 308 and/or one or more position sensors associated withinstrument carriage 306 and/or insertion stage 308 is used to determinethe position L_(x) of proximal point 316 relative to position Lo. Insome examples, position L_(x) may further be used as an indicator of thedistance or insertion depth to which distal end 318 of elongate device310 is inserted into the passageways of the anatomy of patient P.

FIGS. 4A, 4B, 4C, and 4D illustrate the advancement of elongate device310 of FIGS. 3A and 3B through anatomic passageways 402 of the lungs 400of the patient P of FIGS. 1 and 3A and 3B. These passageways 402 includethe trachea and the bronchial tubes. As the elongate device 310 isadvanced with the carriage 306 moving along the insertion stage 308, theoperator O may steer the distal end 318 of elongate device 310 tonavigate through the anatomic passageways 402. In navigating through theanatomic passageways 402, elongate device 310 assumes a shape that maybe measured by the shape sensor 314 extending within the elongate device310.

Referring to FIGS. 5, 6A, GB, 6C, 7, 8, 9, 10, and 11, variousembodiments for image-guided surgical procedure using registrationquality analysis are described. FIG. 5 is a flowchart illustrating ageneral method 500 for use in an image-guided surgical procedure. FIGS.6A, 6B, and 6C illustrate segmentation processes of the general method500 that generates a model of a human lung for registration. FIGS. 7 and8 are flow charts illustrating global registration quality analysis andlocal registration quality analysis respectively. FIGS. 9 and 10illustrate a method for performing registration based the registrationquality analysis. FIG. 11 illustrates a visual representation of theregistration quality indictor. As discussed in detail below, in anexample, providing registration quality indicators to an operator helpthe operator to better understand the uncertainty in the registration,which may further help the operator's navigation operation. In anotherexample, various actions (e.g., by an operator or a control system) maybe performed based on the registration quality indicators, which mayimprove the quality of the image-guided procedure. In yet anotherexample, registration quality analysis may be performed for the entiremodel space or particular regions of interest where registration qualityis of more importance to an operator.

FIG. 5 is a flowchart illustrating a general method 500 for use in animage-guided surgical procedure. The method 500 is illustrated in FIG. 5as a set of operations or processes 502 through 512. Not all of theillustrated processes 502 through 512 may be performed in allembodiments of method 500. Additionally, one or more processes that arenot expressly illustrated in FIG. 5 may be included before, after, inbetween, or as part of the processes 502 through 512. In someembodiments, one or more of the processes may be implemented, at leastin part, in the form of executable code stored on non-transitory,tangible, machine-readable media that when run by one or more processors(e.g., the processors of control system 112) may cause the one or moreprocessors to perform one or more of the processes.

At a process 502, pre-operative or intra-operative image data isobtained from imaging technology such as, computed tomography (CT),magnetic resonance imaging (MRI), fluoroscopy, thermography, ultrasound,optical coherence tomography (OCT), thermal imaging, impedance imaging,laser imaging, or nanotube X-ray imaging. The pre-operative orintra-operative image data may correspond to two-dimensional,three-dimensional, or four-dimensional (including e.g., time based orvelocity based information) images. For example, the image data mayrepresent the human lungs 400 of FIGS. 4A-4D.

At a process 504, a computer system either operating alone or incombination with manual input is used to convert the recorded imagesinto a segmented two-dimensional or three-dimensional compositerepresentation or model of a partial or an entire anatomic organ oranatomic region. For example, FIG. 6A illustrates a segmented model 600of the lungs 400 of FIGS. 4A-4D. Due to limitations in either the dataor segmentation algorithm, the segmented model 600 may not include allof the passageways of interest present within the human lungs, butincludes some passageways 601. For example, relatively narrow and/ordistal passageways of the lungs may not be fully included in thesegmented model 600. The segmented model 600 may be a three-dimensionalmodel, such as a mesh model, linkage model, or another suitable modeldefining the interior lumens or passageways of the lungs. In general,the model serves as a spatial template of the airway geometry within thepre-operative or intra-operative reference frame. The compositerepresentation and the image data set describe the various locations andshapes of the passageways and their connectivity and may omit undesiredportions of the anatomy included in the pre-operative or intra-operativeimage data. In some embodiments, the model 600 may include specificallydesired features, such as a suspected tumor, lesion, or other tissueportion of interest.

During the segmentation process the images are partitioned into segmentsor elements (e.g., pixels or voxels) that share certain characteristicsor computed properties such as color, density, intensity, and texture.This segmentation process results in a two- or three-dimensionalreconstruction that forms a model of the target anatomy based on theobtained image, like the model 600. To represent the model, thesegmentation process may delineate sets of voxels representing thetarget anatomy and then apply a function, such as marching cubefunction, to generate a 3D surface that encloses the voxels. The modelmay be made by generating a mesh, volume, or voxel map. This model maybe shown in the display 110 to aid the operator O in visualizing theanatomy, such as the interior passageways of the lungs.

Additionally or alternatively, the model may include a centerline modelthat includes a set of interconnected line segments or points extendingthrough the centers of the modeled passageways. FIG. 6B shows anexemplary centerline model 602 derived from the model 600 or directlyfrom the imaging data. The centerline segmented model 602 may include aset of three-dimensional straight lines or a set of curved lines thatcorrespond to the approximate center of the passageways contained in thesegmented model 602. The higher the resolution of the model, the moreaccurately the set of straight or curved lines will correspond to thecenter of the passageways. Representing the lungs with the centerlinesegmented model 602 may provide a smaller set of data that is moreefficiently processed by one or more processors or processing cores thanthe data set of the segmented model 602, which represents the walls ofthe passageways of model 600. In this way the functioning of the controlsystem 112 may be improved.

As shown in FIG. 6B, the centerline segmented model 602 includes severalbranch points, some of which are highlighted for visibility in FIG. 6B.The branch points A, B, C, D, and E are shown at each of several of thebranch points. The branch point A may represent the point in the modelat which the trachea divides into the left and right principal bronchi.The right principal bronchus may be identified in the centerline segmentmodel 602 as being located between branch points A and B. Similarly,secondary bronchi are identified by the branch points B and C andbetween the branch points B and E. Another generation may be definedbetween branch points C and D. Each of these generations may beassociated with a representation of the diameter of the lumen of thecorresponding passageway. In some embodiments, the model 602 may includean average diameter value of each segmented generation. The averagediameter value may be a patient-specific value or a more general valuederived from multiple patients.

Where the model includes a centerline model including a set ofinterconnected line segments, those line segments may be converted to acloud or set of points 604, referred to as model points, which arerepresented by the dashed lines of FIG. 6C, By converting the linesegments into points, a desired quantity of model points correspondingto the interconnected line segments can be selected manually orautomatically to represent the centerline model 602. (and thereby themodel 600) during a registration process. In data, each of the points ofthe set of model points 604 may include coordinates such as a set ofX_(M), Y_(M), and Z_(M), coordinates, or other coordinates that identifythe location of each point in the three-dimensional model space. In someembodiments, each of the points may include a generation identifier thatidentifies which passageway generation the points are associated withand/or a diameter or radius value associated with that portion of thecenterline segmented model 602. In some embodiments, informationdescribing the radius or diameter associated with a given point may beprovided as part of a separate data set.

After the centerline segmented model 602 is generated and stored in dataas the set of points 604 shown in FIG. 6C, the model points 604 may beretrieved from data storage for use in an image-guided surgicalprocedure. In order to use the centerline segmented model 602 and themodel 600 in the image-guided surgical procedure, the model points 604may be registered to associate the modeled passageways in the model 600with the patient's actual anatomy as present in a surgical environment.

Returning to FIG. 5 , at a process 506, measured points may be obtainedfrom patient anatomy that correspond to the anatomical model, asdescribed with reference to FIGS. 3A-B and 4A-D. Measured points may begenerated by advancing an elongate device through anatomy and/or tolandmarks in the anatomy, while measuring the position of a distal endof the elongate device or pose of the elongate device using a sensorsystem (e.g., the sensor system 108). The measured points are associatedwith a patient, space, and may also be referred to as patient, spacepoints.

At a process 508, the anatomic model data of a model space is registeredto the patient anatomy of a patient space (or vice versa) prior toand/or during the course of an image-guided surgical procedure on thepatient. Generally, registration involves the matching of measured pointto model points of the model through the use of rigid and/or non-rigidtransforms. A point set registration method (e.g., iterative closestpoint (ICP) technique) may also be used in registration processes withinthe scope of this disclosure. Such a point set registration method maygenerate a transformation that aligns the measured points (also referredto as a measured point set) and the model points (also referred to as amodel point set).

In various embodiments, the quality of the registration may depend onvarious factors, including for example, the numbers of the measuredpoints and/or model points, the density of the measured points and/ormodel points, the distribution of the measured points and/or modelpoints relative to a region of interest, measurement errors associatedwith the measured points and/or model points, and deformation of thepatient anatomy associated with the measured points and/or model points.

At a process 510, a registration quality analysis may be performed togenerate a registration quality indicator for the registration performedat the process 508. Such a registration quality indicator may beprovided to the operator (e.g., using a display system 110), which mayprovide an operator a degree of confidence in the registration quality.In an example where the registration quality indicator indicates thatthe registration quality is high (e.g., based on a high qualitythreshold), such a registration quality indicator may bolster theoperator's confidence in the image-guided surgical procedure. In anexample where the registration quality indicator indicates that theregistration quality is low (e.g., based on a low quality threshold),such a registration quality indicator communicates uncertainty (e.g.,dubious alignment or erroneous alignment) to the operator.

At a process 512, various actions may be performed based on theregistration quality indicator. For example, when the registrationquality indicator indicates that the registration quality is low (e.g.,based on a low quality threshold), instructions may be provided to theoperator (e.g., using the display system 110) for taking necessaryprecautions such as cease advancing the elongate device. In anotherexample, when the registration quality indicator indicates that theregistration quality is low, the operator may be directed to useadditional/alternative navigational aids (e.g. fluoroscopy orintra-operative CT). In yet another example, when the registrationquality indicator indicates that the registration quality is low, aregistration at process 508 may be performed again to improveregistration quality or the operator may be instructed to perform theregistration process again.

Referring to FIGS. 7, and 8 , the process for registration qualityanalysis (e.g., process 510 of FIG. 5 ) may include a globalregistration quality analysis, a local registration quality analysis,and/or a combination thereof. FIG. 7 illustrates a method 700 for globalregistration quality analysis, and FIG. 8 illustrates a method 800 forlocal registration quality analysis.

Referring to FIG. 7 , the method 700 in FIG. 7 is illustrated as a setof operations or processes 702 through 716. Not all of the illustratedprocesses 702 through 716 may be performed in all embodiments of method700. Additionally, one or more processes that are not expresslyillustrated in FIG. 7 may be included before, after, in between, or aspart of the processes 702 through 716. In some embodiments, one or moreof the processes may be implemented, at least in part, in the form ofexecutable code stored on non-transitory, tangible, machine-readablemedia that when run by one or more processors (e.g., the processors ofcontrol system 112) may cause the one or more processors to perform oneor more of the processes.

The method 700 begins at a process 702, where a global registration isperformed to register the model space to the patient space. The globalregistration may generate an optimal transformation T_(opt) for aligningthe model point set of the model space and the measured point set of thepatient space. In such a global registration, a registration algorithm(e.g., a point set registration algorithm) may be used to register allthe points in both the measured point set and the model point set in aglobal registration, an optimal transformation T_(opt) is computed fromboth the points inside and outside such particular regions of interest.It is noted that in some examples, the optimal transformation T_(opt) isa solution computed based on an optimal set of input parameters, whichmay be different from the absolute or true optimal solution.

In some embodiments, at process 702, a registration algorithm for theglobal registration may use a set of optimal initial parameters (e.g.best guess initial parameters, initial parameters generated based onparameterization) for generating the optimal transformation T. Theinitial parameters may include a transformation seed or, a weightingscheme applied to either the model or collected points, a pointcorrespondence configuration (e.g., soft correspondence, hardcorrespondence), and/or other suitable initial parameters. In someembodiments, the process 702 is skipped. In such embodiments, atransformation provided by process 508 of FIG. 5 may be used as thetransformation T_(opt), and the initial parameters for the registrationperformed at process 508 may be used as the optimal initial parameters.

The method 700 continues to a process 704, where a perturbation index iis initialized to zero. The method 700 may then continue to a process706, where the set of optimal input parameters for generating theoptimal transformation T_(opt) is perturbed to generate the set ofperturbed input parameters. One or more of the parameters in the set ofoptimal input parameters, e.g., the transformation seed, the pointweighting scheme, the correspondence configuration, may be perturbedslightly to generate the i^(th) set of perturbed initial parameters.

The method 700 may then proceed to the process 708, where a globalregistration is performed using the i^(th) set of perturbed initialparameters to generate a perturbed transformation T_(perturbed) ^(i). Inan example, in the process 708, a registration algorithm (e.g., a pointset registration algorithm) is restarted to generate the perturbedtransformation T_(perturbed) ^(i) using the set of perturbed initialparameters.

The method 700 may then proceed to a process 710, where it is determinedwhether perturbation index i is less than n-1, where n is apredetermined total perturbation number. The total perturbation number nmay be an integer having a value equal to or greater than one. In someembodiments, the total perturbation number n and the modification valuefor each initial parameter may be configured such that a distribution ofregistration transformations T_(perturbed) ⁰ through T_(perturbed)^(n-1) generated by the n perturbations is representative of uncertaintyin the optimal transformation T_(opt).

In various embodiments, n sets of perturbed initial parameters aregenerated for the n perturbations respectively. The n sets of perturbedinitial parameters are generated by modifying the optimal initialparameters such that those perturbed initial parameters cover anexpected range of initial parameter values. In an example, the n sets ofperturbed initial parameters include random initial parameters valueswithin that expected range of initial parameter values. The registrationseed transform may be perturbed. In another example airwayregistration), a perturbation (e.g. a perturbation of up to 2 cm and 30degrees) might be applied to the initial transform. The magnitude: ofthe perturbation maybe determined to have a value that is sufficientlylarge such that it is reflects the uncertainty in the initial guess andis sufficiently small so that solutions that are outrageous or outsidethe scope of reality are avoided. In another embodiment the weighting ofmodel or collected points may be adjusted either randomly or inassociation with a spatial distribution. For example, the pointweighting may be perturbed to favor points in one side of the lung overanother side of the lung, or to favor points in a given lobe overanother lobe. In another embodiment, the algorithm parameters may bealtered such that correspondences are computed differently or weighteddifferently. In an example, by using a particular set of algorithmparameters, registration may be driven heavily by point correspondencesthat are poorly matched (e.g. points that do not align well). In anotherexample, by using another particular set of algorithm parameters, pointcorrespondences with large errors might be down-weighted or ignoredaltogether.

In some embodiments, in the process 710, it is determined that theperturbation index i is less than n-1. In those embodiments, the method700 proceeds to a process 712 to increase the perturbation index i byone. The method 700 may then proceed to the processes 706 and 708 tocompute the next perturbed transformation T_(perturbed) ^(i) with thei^(th) set of perturbed initial parameters.

In some embodiments, at the process 710, it is determined that theperturbation index i is not less than n−1. In those embodiments, themethod 700 proceeds to a process 714, where for each point y^(k) of aset of points in the model space, a point mean registration error E^(k)is computed as follows:

$\begin{matrix}{{E^{\overset{˙}{\kappa}} = {\frac{1}{n}{\sum_{i = 0}^{n - 1}{{y^{k} - {T_{perturbed}^{i}T_{opt}^{- 1}y^{k}}}}}}},} & (1)\end{matrix}$

where k is a point index, i is a perturbation index, and n is the totalnumber of perturbations. In various embodiments, the error E^(k) may becomputed in any other ways to measure the back projection error givenslightly different estimates of a rigid transformation T_(opt) andT_(perturbed) The model point y^(k) may be populated from various setsof points, including for example, all points used in the model, pointswithin a region of interest, points within some grid spaced equallywithin the entire volume, any other suitable set of points, and/or acombination thereof.

In some embodiments, the point mean registration error is computed foreach measured point of a set of measured points in the patient space. Inan example, by computing the point mean registration errors for themeasured points in the patient space, the registration quality indictormay be computed to indicate uncertainty related to the path that anoperator drives the elongate device 202

The method 700 may then proceed to a process 716, where a globalregistration quality index (GRQI) is computed based on the point meanregistration errors E⁰ through E^(m-1) follows:

$\begin{matrix}{{{GRQI} = {\frac{1}{m}{\sum_{k = 0}^{m - 1}E^{k}}}},} & (2)\end{matrix}$

where k is a point index, and in is the total number of points (e.g., aportion of points or all points in the model space or patient space)used for computing the point mean registration errors. Note that whileequations (1) and (2) are used to as an example to illustrate computinga GRQI, any suitable algorithms for computing a global registrationquality index may be used.

In some embodiments, a global registration quality indicator includesthe GRQI. In some embodiments, the global registration quality indicatoris determined based on the GRQI. For example, the global registrationquality indicator may include a quality level having a value selectedfrom “LOW,” “MEDIUM,” and “HIGH” determined based on GRQI. In an examplewhere the GRQI is less than a global registration quality highthreshold, the global registration quality indicator includes a qualitylevel “HIGH.” In another example where the GRQI is greater than a globalregistration quality low threshold, the global registration qualityindicator includes a quality level “LOW.” Yet in another example wherethe GRQI is between the global registration quality high threshold andglobal registration quality low threshold, the global registrationquality indicator includes a quality level “MEDIUM.”

While the method 700 of FIG. 7 for global registration quality analysiscomputes a distribution of registration transformations that span arange of expected initial parameters, those global registrationtransformations may be insensitive to misalignment of a subset ofpoints, e.g., in a region of interest. Such a region of interest may bea region where a biopsy is performed by the operator to extract tissueor a region where a therapy is applied by the operator. As such, thelocal registration quality of a region of interest may be more importantto the operator than the global registration quality. Referring to FIG.8 , illustrated is a method 800 for local registration quality analysis,which may provide a registration quality indicator associated with suchsmaller subset of points in a region of interest. In some embodiments, alocal registration quality indicator may be computed by comparingconsistency in registration transformation results associated with twoor more regions in the model space.

The method 800 in FIG. 8 is illustrated as a set of operations orprocesses 802 through 810. Not all of the illustrated processes 802through 810 may be performed in all embodiments of method 800.Additionally, one or more processes that are not expressly illustratedin FIG. 8 may be included before, after, in between, or as part of theprocesses 802 through 810. In some embodiments, one or more of theprocesses may be implemented, at least in part, in the form ofexecutable code stored on non-transitory, tangible, machine-readablemedia that when run by one or more processors (e.g., the processors ofcontrol system 112) may cause the one or more processors to perform oneor more of the processes.

The method 800 begins at a process 802, where a local registration isperformed to register a region of interest of a model space to thepatient space and generate a region of interest transformation T¹. Sucha region of interest transformation T¹ may be computed using modelpoints inside the region of interest of the model space. In someembodiments, the region of interest is provided by the operator. In someembodiments, at this stage, a global registration (e.g., process 508 ofFIG. 5 ) has been performed using a set of optimal initial parameters togenerating the transformation T_(opt). The local registration toregister the region of interest may be performed using that set ofoptimal initial parameters or a set of perturbed initial parametersgenerated based on a set of optimal initial parameters.

The method 800 may then proceed to a process 804, where a comparisonregion of the model space is determined. In some embodiments, thecomparison region is determined based on the region of interest. Forexample, the comparison region may be the model space excluding theregion of interest. In some embodiments, the comparison region is aregion provided by the operator. In an example, the region of interestand comparison region may overlap. In another example, the region ofinterest and comparison region do not overlap. In yet another example,the region of interest and comparison region are not adjacent to eachother. In yet another example, the region of interest and comparisonregions may be fuzzy such that some points may be partial members ofboth regions. In an example for registration of points within theairways, the region of interest may consist of all points within a givenlobe, while the comparison region is all points outside the lobe. Inanother example, the comparison region may include all points not withina lobe, but within the main stem bronchi. In yet another example, thecomparison region may consist of points within an alternative lobe ormultiple lobes.

The method 800 may then proceed to a process 806, where a localregistration to register the comparison region of the model space to thepatient space is performed to compute a comparison region transformationT₂ for the comparison region, Such a comparison region transformation T₂may be computed only using points inside the comparison region of themodel space. In some embodiments, the local registration to register thecomparison region uses a set of initial parameters that is the same asthe set of initial parameters used at the process 802 to register theregion of interest.

The method 800 may then proceed to a process 808, where for each pointy^(k1) of the points within the region of interest, a point registrationerror E^(k1) is computed based on the region of interest transformationT₁ and comparison region transformation T₂ as follows:

E ^(k1) =∥T ₁ ⁻¹ y ^(k1) −T ₂ ⁻¹ y ^(k1)∥  (3)

where k1 is a point index of the points in the region of interest.

In some embodiments, processes 802 through 808 are repeated for n1perturbations to generate perturbed transformations (T₁ ⁰)⁻¹ through (T₁^(n1-1))⁻¹ and (T₂ ⁰)⁻¹ through (T₂ ^(n1-1))⁻¹ for the region ofinterest and the comparison region respectively using n1 sets ofperturbed initial parameters. Such perturbed initial parameters may begenerated substantially similar to the perturbed initial parametersdescribed above in FIG. 7 . In some embodiments, in each of the n1perturbations, the same set of perturbed initial parameters may be usedto register the region of interest and comparison region to generate T₁^(i) and T₂ ^(i) respectively, wherein i is a perturbation index between0 and n1-1. A point registration error E^(k1) may be computed as a pointmean registration error based on the perturbed transformations (T₁ ⁰)⁻¹through (T₁ ^(n1-1))⁻¹ and (T₂ ⁰)⁻¹ through (T₂ ^(n1-1))⁻¹ as follows:

$\begin{matrix}{{E^{k1} = {\frac{1}{n1}{\sum_{i = 0}^{{n1} - 1}{{{\left( T_{1}^{i} \right)^{- 1}y^{k1}} - {\left( T_{2}^{i} \right)^{- 1}y^{k1}}}}}}},} & (4)\end{matrix}$

where k1 is a point index of the points in the region of interest, i isa perturbation index, and n1 is the total number of perturbations forlocal registration quality analysis.

The method 800 may then proceed to a process 810, where a localregistration quality index (LRQI) is computed using the mean error overall ml points in the region of interest of the model space as follows:

$\begin{matrix}{{{LRQI} = {\frac{1}{m1}{\sum_{{k1} = 0}^{{m1} - 1}E^{k1}}}},} & (5)\end{matrix}$

where k1 is the point index of the region of interest, and ml is thetotal number of points in the region of interest. Note that whileequations (3), (4), and (5) are used to as an example to illustratecomputing a LRQI, any suitable algorithms for computing a localregistration quality index may be used.

In some embodiments, a local registration quality indicator includes theLRQI, In some embodiments, the local registration quality indicator isdetermined based on the LRQI. For example, the local registrationquality indicator may include a quality level having a value selectedfrom “LOW,” “MEDIUM,” and “HIGH” determined based on LRQI. In an examplewhere the LRQI is less than a local registration quality high threshold,the local registration quality indicator includes a quality level“HIGH.” In another example where the LRQI is greater than a localregistration quality low threshold, the local registration qualityindicator includes a quality level “LOW.” Yet in another example wherethe LRQI is between the local registration quality high threshold andlocal registration quality low threshold, the local registration qualityindicator includes a quality level “MEDIUM.” In some embodiments, suchlocal/global registration quality high/low thresholds may be configuredby the operator for a particular region of interest.

Referring to FIGS. 9 and 10 , in some embodiments, actions performed(e.g., in process 512 of FIG. 5 ) based on the registration qualityindicator may include performing registration again (re-registration) toimprove the registration quality. Such a re-registration may beperformed in a modified model space by excluding particular regions fromthe model space based on the registration quality indicators (e.g., a glocal registration quality indicator and one or more local registrationquality indicators).

The method 900 in FIG. 9 is illustrated as a set of operations orprocesses 902 through 910. Not all of the illustrated processes 902through 910 may be performed in all embodiments of method 900.Additionally, one or more processes that are not expressly illustratedin FIG. 9 may be included before, after, in between, or as part of theprocesses 902 through 910. In some embodiments, one or more of theprocesses may be implemented, at least in part, in the form ofexecutable code stored on non-transitory, tangible, machine-readablemedia that when run by one or more processors (e.g., the processors ofcontrol system 112) may cause the one or more processors to perform oneor more of the processes.

The method 900 begins at a process 902, where a first set of regions ofthe model space and corresponding local registration quality thresholdsare received. The first set of regions may include one or more regionsof interest identified by the operator.

Referring to FIG. 10 , illustrated therein is an exemplary visualrepresentation to an operator. As illustrated in FIG. 10 , a display 110displays, in a user interface, a rendering of anatomic passageways of ahuman lung based upon anatomic model 600 of FIG. 6A. With the modelspace registered to the patient as described above in FIG. 5 (e.g., atprocess 508 of FIG. 5 ), the current shape of the elongate device 310and the location of the distal end 318 may be located and displayedconcurrently with the rendering of the passageways 601, which includespassageways 601A and 601B. The model points may be visually representedin a user interface by a plurality of points or dots on the display oras a rendered model, such as a mesh or wire model created based on theset of data points.

As illustrated in FIG. 10 , in some embodiments, at a process 902, a setof regions including regions 1002 and 1004 are received. In an example,the regions 1002 and 1004 have the same registration quality thresholds.In another example, the regions 1002 and 1004 have differentregistration quality thresholds (e.g., based on their registrationquality importance for an operator).

In some embodiments, at this stage of process 902, a global registrationto register the entire model space to the patient space has already beenperformed to generate a transformation T_(opt). In the example of FIG.10 , a global registration quality analysis has been performed, and thedisplay 110 includes a global registration quality area 1006 displayingthe global registration quality indicator (e. “HIGH”) for the currentglobal registration.

The method 900 may proceed to a process 904, where a local registrationquality analysis (e.g., a method 800 of FIG. 8 ) may be performed foreach region in the first set of regions. A local registration qualityindicator may be generated for each region (e.g., using thecorresponding local registration quality thresholds). In someembodiments, the local registration quality analysis receives a set ofoptimal initial parameters that is used for generating thetransformation T_(opt), for the global registration, and uses that setof optimal initial parameters and/or one or more sets of perturbedinitial parameters generated based on that set of optimal initialparameters for performing registration for the regions as discussedabove with respect to FIG. 8 .

As illustrated in FIG. 10 , in some embodiments, at the process 904, thedisplay 110 may include a local registration quality area 1008displaying the local registration quality indicators (e.g., “MEDIUM” forregion 1002, “LOW” for region 1004) for the first set of regions that isgenerated at the process 904.

The method 900 may proceed to a process 906, where one or more regionsto exclude are determined based on the global registration qualityindicator and/or local registration quality indicators. In an example, acontrol system may determine the one or more regions to exclude (e.g.,based on registration quality thresholds). In another example, asillustrated in FIG. 10 , an operator may provide the one or more regionsto exclude. In the example of FIG. 10 , the display 110 includes are-register area 1010, and the operator determines to not exclude region1002 in the re-registration (e.g., using a choice 1012), but to excluderegion 1004 in the re-registration (e.g., using a choice 1014).

The method 900 may proceed to a process 908 to generate a modified modelspace based on the one or more regions to exclude. In the example ofFIG. 10 , at a process 908, a modified model space is generated byexcluding the region 1004 from the model space.

The method 900 may proceed to a process 910 to perform re-registrationusing the modified model space. In an example, such a re-registration isperformed automatically by a control system based on the registrationquality indicators and predetermined re-registration criteria. Inanother example, as illustrated in FIG. 10 , an operator may start there-registration with the modified model space by selecting the button1016. While in some embodiments the operator may perform re-registrationbased on the local registration quality indicators associated withregions of interest, in other embodiments, the operator may performre-registration based on the global registration quality indicator. Byperforming a re-registration using the modified model space, theregistration quality (for the entire model space and/or particularregions) may be improved.

Referring to FIG. 11 , in various embodiments, registration qualityindicators may be displayed to an operator using various visualrepresentations, including for example, number representations,graphical representations such as heat map using colors, shades,patterns, any suitable visual representation, and/or a combinationthereof. As shown in the example of FIG. 11 , a display 110 displays, ina user interface, a rendering of anatomic passageways of a human lungbased upon anatomic model 600 of FIG. 6A, where the current shape of theelongate device 310 and the location of the distal end 318 are locatedand displayed concurrently with the rendering of the passageways 601.Further, a heat map (e.g., using shades, colors, patterns, etc.) basedon the registration quality indicator is displayed in the anatomic model600. For example, points in the region 1102 of the anatomic model 600have registration quality indicator values indicating low quality (e.g.,by comparing with one or more low registration quality thresholds). Theregion 1102 may have a first shade (e.g., light) or a first color (e.g.,orange) associated with low quality. Points in the region 1104 haveregistration quality indicator values indicating medium quality (e.g.,by comparing with one or more medium registration quality thresholds).As such, the region 1104 may have a second shade (e.g., medium) or asecond color (e.g., yellow) associated with medium quality that isdifferent from the first shade or the first color. Points in the region1106 have registration quality indicator values indicating high quality(e.g., by comparing with one or more high registration qualitythresholds). As such, the region 1106 may have a third shade (e.g.,dark) or a third color (e.g., green) different from the first and secondshades or the first and second colors. In various embodiments, anoperator may accept the registration (e.g., by selecting button 1108“Accept Registration”) or restart the registration (by selecting button(e.g., by selecting button 1110 “Restart Registration”) based on thevisual representation of the registration quality indicators in thedisplay 110.

The systems and methods of this disclosure may be used for connectedbronchial passageways of the lung. The systems and methods may also besuited for navigation and treatment of other tissues, via natural orsurgically created connected passageways, in any of a variety ofanatomical systems including the colon, the intestines, the kidneys, thebrain, the heart, the circulatory system, or the like. The systems andmethods may also be suitable for navigation around the traceable surfaceof an organ. The methods and embodiments of this disclosure are alsosuitable for non-surgical applications.

One or more elements in embodiments of the invention may be implementedin software to execute on a processor of a computer system such ascontrol system 112. When implemented in software, the elements of theembodiments of the invention are essentially the code segments toperform the necessary tasks. The program or code segments can be storedin a processor readable storage medium or device that may have beendownloaded by way of a computer data signal embodied in a carrier waveover a transmission medium or a communication link. The processorreadable storage device may include any medium that can storeinformation including an optical medium, semiconductor medium, andmagnetic medium. Processor readable storage device examples include anelectronic circuit; a semiconductor device, a semiconductor memorydevice, a read only memory (ROM), a flash memory, an erasableprogrammable read only memory (EPROM); a floppy diskette, a CD-ROM, anoptical disk, a hard disk, or other storage device. The code segmentsmay be downloaded via computer networks such as the Internet, Intranet,etc.

Note that the processes and displays presented may not inherently berelated to any particular computer or other apparatus. Variousgeneral-purpose systems may be used with programs in accordance with theteachings herein, or it may prove convenient to construct a morespecialized apparatus to perform the operations described. The requiredstructure for a variety of these systems will appear as elements in theclaims. In addition, the embodiments of the invention are not describedwith reference to any particular programming language. It will beappreciated that a variety of programming languages may be used toimplement the teachings of the invention as described herein.

While certain exemplary embodiments of the invention have been describedand shown in the accompanying drawings, it is to be understood that suchembodiments are merely illustrative of and not restrictive on the broadinvention, and that the embodiments of the invention not be limited tothe specific constructions and arrangements shown and described, sincevarious other modifications may occur to those ordinarily skilled in theart.

1-45. (canceled)
 46. A system comprising: a non-transitory memory; oneor more processors coupled to the non-transitory memory and configuredto read instructions to cause the system to perform operationscomprising: receiving a set of model points of a model of an anatomicstructure of a patient, the model points being associated with a modelspace; receiving, from a sensor system of an elongate device, a set ofmeasured points of the anatomic structure of the patient, the measuredpoints being associated with a patient space; registering the set ofmodel points to the set of measured points using a set of initialparameters to generate a first transformation; perturbing the set ofinitial parameters to generate one or more sets of perturbed initialparameters; registering the set of model points to the set of measuredpoints using the one or more sets of perturbed initial parameters togenerate one or more perturbed transformations; and generating aregistration quality indicator based on the first transformation and theone or more perturbed transformations.
 47. The system of claim 46,wherein each set of the one or more sets of perturbed initial parametersis within a predetermined range of initial parameter values, and whereinthe predetermined range is associated with uncertainty of the firsttransformation.
 48. The system of claim 47, wherein each set of the oneor more sets of perturbed initial parameters includes random initialparameter values within the predetermined range.
 49. The system of claim46, wherein a first set of perturbed initial parameters includes aperturbed transformation seed that is different from a transformationseed of the set of initial parameters.
 50. The system of claim 46,wherein a first set of perturbed initial parameters includes a perturbedpoint weighting scheme that is different from a point weighting schemeof the set of initial parameters.
 51. The system of claim 46, wherein afirst set of perturbed initial parameters includes a perturbedcorrespondence configuration different from a correspondenceconfiguration of the set of initial parameters.
 52. The system of claim46, wherein the generating the registration quality indicator includes:generating a plurality of point registration errors associated with theanatomic structure based on the first transformation and the one or moreperturbed transformations; and generating a registration quality indexof the registration quality indicator by computing a mean error of theplurality of point registration errors.
 53. The system of claim 52,wherein the generating the plurality of point registration errorsincludes: generating one or more perturbed point errors corresponding tothe one or more perturbed transformations; and computing a mean error ofthe one or more perturbed point errors.
 54. The system of claim 52,wherein the plurality of point registration errors is associated with atarget region of the anatomic structure.
 55. The system of claim 46,wherein the operations further comprise: comparing the registrationquality indicator with a registration quality threshold to generate acomparison result.
 56. A non-transitory machine-readable mediumcomprising a plurality of machine-readable instructions which, whenexecuted by one or more processors, are adapted to cause the one or moreprocessors to perform a method comprising: receiving a set of modelpoints of a model of an anatomic structure of a patient, the modelpoints being associated with a model space; receiving, from a sensorsystem of an elongate device, a set of measured points of the anatomicstructure of the patient, the measured points being associated with apatient space; registering the set of model points to the set ofmeasured points using a set of initial parameters to generate a firsttransformation; perturbing the set of initial parameters to generate oneor more sets of perturbed initial parameters; registering the set ofmodel points to the set of measured points using the one or more sets ofperturbed initial parameters to generate one or more perturbedtransformations; and generating a registration quality indicator basedon the first transformation and the one or more perturbedtransformations.
 57. The non-transitory machine-readable medium of claim56, wherein each set of the one or more sets of perturbed initialparameters is within a predetermined range of initial parameter values,and wherein the predetermined range is associated with uncertainty ofthe first transformation.
 58. The non-transitory machine-readable mediumof claim 57, wherein each set of the one or more sets of perturbedinitial parameters includes random initial parameter values within thepredetermined range.
 59. The non-transitory machine-readable medium ofclaim 56, wherein a first set of perturbed initial parameters includes aperturbed transformation seed that is different from a transformationseed of the set of initial parameters.
 60. The non-transitorymachine-readable medium of claim 56, wherein a first set of perturbedinitial parameters includes a perturbed point weighting scheme that isdifferent from a point weighting scheme of the set of initialparameters.
 61. The non-transitory machine-readable medium of claim 56,wherein a first set of perturbed initial parameters includes a perturbedcorrespondence configuration different from a correspondenceconfiguration of the set of initial parameters.
 62. The non-transitorymachine-readable medium of claim 56, wherein the generating theregistration quality indicator includes: generating a plurality of pointregistration errors associated with the anatomic structure based on thefirst transformation and the one or more perturbed transformations; andgenerating a registration quality index of the registration qualityindicator by computing a mean error of the plurality of pointregistration errors.
 63. The non-transitory machine-readable medium ofclaim 62, wherein the generating the plurality of point registrationerrors includes: generating one or more perturbed point errorscorresponding to the one or more perturbed transformations; andcomputing a mean error of the one or more perturbed point errors. 64.The non-transitory machine-readable medium of claim 62, wherein theplurality of point registration errors is associated with a targetregion of the anatomic structure.
 65. The non-transitorymachine-readable medium of claim 56, wherein the method furthercomprises: comparing the registration quality indicator with aregistration quality threshold to generate a comparison result.